In many applications a determination of the position and orientation of a target object in 3D space is required. Therein the spatial posture of a target object in space by measuring its spatial coordinates and its spatial orientation has to be determined, resulting in up to six degrees of freedom which needs to be evaluated. The target object can be the object of interest itself or a dedicated target object which is attached thereto.
For example, in WO 2005/039836 a robot arm positioning control by observing position and orientation in 3D space of a working tool attached to the robots arm by a camera system is presented.
Controlling of construction machinery is another application, e.g. as described in U.S. Pat. No. 5,771,978, wherein a Tracking-Station is tracking a retro reflector as a target mark which is attached to a working tool of the machine, in this example a blade of a dozer for moving earth into a desired shape.
The document EP 1 171 752 (—honoured with the EPO inventors award in 2010—) describes another application, wherein the position and orientation of a measurement probe for coordinate measurement is determined in six degrees of freedom by a tracker-device, which is tracking multiple dedicated discrete singular reference points that are attached to the probe for determining the posture of the measuring probe-tip in six degrees of freedom.
The above mentioned methods of a high accuracy posture determination of a target object require rather complicate and expensive surveying apparatus, e.g. devices such as Laser-Scanners, Total Stations, Tracking Stations, etc.
Alternatives, such as stereographic imaging, require complex multiple camera and illumination setups and are of comparably low accuracy and suffer from ambiguity problems in measurement.
Pattern projection systems, like the DAVID laser scanner project from TU-Braunschweig are also known. As e.g. published on www.david-laserscanner.com, a laser pattern, e.g. a line, is scanned over a target and a digital video camera records the resulting images, whereof a 3D-point-cloud-model of the scanned object is created. A photographic image of the camera can also be used to texture the resulting 3D-Model. Due to the scanning, a digitalisation of the full field of view of the camera takes time and is therefore improper for evaluating or observing non-steady objects.
In Microsoft's technical report on camera calibration, published in the “IEEE transactions on pattern analysis and machine intelligence”, Vol. 22, No. 11, November 2000, a calibration systems for digital picture cameras for 3D computer vision is described, which observes a single planar pattern at different orientations to achieve a camera calibration to allow a more accurate extraction of metric information from 2D images taken by the camera.
In US 2005/0002555 a first set of stored images of a workpiece—taken from different directions—are compared to an actual image of a workpiece. The camera is the moved to closely resemble the stored image of the first set and a second comparison with a second set of stored images—taken with a narrower pitch—is done to determine the position and attitude of the workpiece.
For a 3D measurement by gathering 3D point cloud, also the usage of Range Imaging Modules (RIM) is a known technique, which can digitize a whole scenery in “one shot”. A RIM-camera comprises in principle an array of pixels functioning as optoelectronic distance meters, e.g. based on a time of flight measurement (TOF) of optical radiation, e.g. by pulse, phase, signal-shape or interferometrical principles which are known in the art, e.g. from “Electronic Distance Measurement” from J. M. Rüeger, Ed. 4, Springer-Verlag, Berlin, Heidelberg 1996. A range image taken by such a RIM-camera comprises distance information for each of the pixels of the camera, resulting in a 3D image of the scenery taken from a single point of view. The measurement is thereby done in polar coordinates according to the angular field of view of a camera-pixel and the therewith determined distance in a unit of length.
A graphical view of the measured RIM-image can e.g. be presented in a two dimensional representation by luminescence—or colour keying of the distances into the 2D picture of the view of the RIM-image. Alternatively, it can be presented in an axonometric view or by a real 3D display. In addition to the range information, also an intensity of the scattered back distance measurement radiation can be determined by the RIM camera an provided in an image.
Nevertheless, the available RIM-devices are often suffering from relatively low resolutions and accuracies, e.g. a resolution of the field of view are presently well below 1 Mega pixels (e.g. 176×144 pixels=0.025 Mega-Pixels) and a distance resolution in the cm range is a common value. There are certain known techniques to improve those parameters, like zooming and angular movement of the field of view, but they all suffer from drawbacks like again prolonging the measurement time which is an important parameter, in particular in the case of measuring a potentially moving target.
For example, WO 2006/075017 presents a method and geodetic apparatus for surveying at least one target. Therein, a range imaging module (RIM) comprising sensors in a matrix arrangement, e.g. 32×32 sensors are used for providing a range image. The range image provides a so called cluster of points or point cloud information comprising the range of the target points imaged by the respective pixels of the sensor. In order to improve the accuracy of the range image, range images of details subsequently reduced in size can be taken. However, although this may improve the accuracy of the range image in a certain manner, due to the relative low resolution of the range image, it is still difficult to exactly address distinctive target points of the target which can be extremely important in case of smaller and/or moving targets which are changing their orientation and position while being surveyed. Another drawback is the prolonged measurement time for measuring the full scenario with the respective high resolution and accuracy.
In CN 102252653, a TOF camera is used and three identifiable objects from a coordinate information database of target objects are selected as mark points for position and attitude parameter determination.
As mentioned, a general drawback of RIM-Imaging often is the low image resolution of the available RIM-cameras in particular compared to the state of the art digital photographic cameras having resolutions of up to tens of Mega-Pixels and more.
The proceedings of “3DPVT'08—the fourth international symposium on 3D data processing, visualization and transmission”—at Georgia Institute of Technology, June 2008, teaches the usage of edges or silhouette cues from a 2D image together with RIM-Camera data for enhancing the 3D reconstruction of an object. Therein, multiple cameras are used for taking RIM images and video pictures from multiple points of view, whereof a 3D model can be derived—also of concave surfaces which are otherwise known to be quite difficult to handle by stereographic imaging.
The system of US 2010/0046802 describes an approach which is using a combination of a RIM-camera and a picture camera for enhancing 3D resolution for an enhanced depth feel for a movie or still camera by the presented distance estimation apparatus. The document comprises different aspects and embodiments of such an approach and can serve as a reference for some of the underlying principles of the present invention. In particular edge-extraction techniques and other aspects of matching a range image and a corresponding visual picture are elaborated therein.
The prior art designs are either requiring complicated and expensive measurement apparatus or are of low accuracy or requiring long measurement times.
Some embodiments of the present invention may provide for an improved method and apparatus to determine position and orientation of an object in a viewed scenery, in particular in 3D space in six degrees of freedom.
Some embodiments of the present invention may achieve high accuracy in the six degrees of freedom measurement of a target object while keeping the measurement setup simple and preferably to do the measurement by a single apparatus from a single point of view.
Some embodiments of the present invention may provide for an improved position and orientation determination with a short measurement time.
Some embodiments of the invention may provide a method for determining the position and orientation in six degrees of 15 freedom of an object of known shape inside of an evaluated scenery with high accuracy.
Some embodiments provide for a 6-DOF measurement with a reduced measurement time, in particular to allow measurements of moving objects.
Some embodiments of the invention may provide a method of 3D measurement which can in particular be used to precisely measure a distinct point of the measurement object, preferably by tactile means, and thereby enhance a recorded point cloud in positional measurement accuracy.
Some embodiments of the invention may also provide a measurement method which is capable of measuring parts of the measurement object which are shaded from the measurement devices point of view.